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The purpose of this study vas to pre.^jent and evaluate 
Bayesian-type models for estimating probabilities oi yrogvam 
completion and for predicting first quarter grade point averages of 
community college students entering certain allied health fields. Tvo 
Bayesian models were tested. Bayesian Hodel 1~Estimating 
Probabilities of Program Completion — vas developed from discrete 
cases of Bayes' formula vith counselors' inputs as a priori 
probabilities and posterior probabilities of graduate status of the 
discriminant analysis function as likelihoods. The a priori 
probabilities and likelihood probabilities vere combined in Bayes* 
Theorem to produce posterior probabilities of succe^ssf ul program 
completion. Bayesian Hodel 2— Predicting First Quarter Grade Point 
Averages — vhich vas an application and specialization of the Bayesian 
linear model developed by Lindley and Smith, involved the assumption 
of homogeneity of regression coefficients (but not intercepts) across 
groups. The efficiencies for both Bayesian models vere compared a|id 
evaluated in terms of tvo counselor-selection models and tvo 
classical statistical models vith the results shoving that Bayesian 
Model 1 vas most efficient in selecting successful students vhile no 
significant difference in efficiency vas found for Bayesian Hodel 2 
as compared to a classical statistical model. (Author/JCS) 



ED 136 855 

AOTHOB 
TITLE 



POB DATE 
HOTE 



EDES PEICE 
DESCBIPTOBS 



ABSTBACT 



* Documents acquired by EBIC include many informal unpublished * 

* materials not availeible from other sources. EEIC meikes every effort * 

* to obtain the best copy available. Nevertheless, items of margipal * 

* reproducibility are often encountered and this affects the quality * 

* of the microfiche and hardcopy reproductions EBIC makes available * 

* via the EEIC Document Eeproduction Service (EDBS) . EDES is not * 

* responsibJi^e for the quality of the original document. Eeproductions * 

* supplied by EDBS are the best that can be made from the original. * 



EKLC 



US OEPARTMENTOFHEALTa 
E0UCA.T10N& WELFARE 
NATIONAL INSTITUTE OF 
EDUCATION 

THIS DOCUMENT MAS BEEN REPRO- 
DUCED EXACTLY AS RECEIVED FROM 
TME PERSON OR ORGANIZATION ORIGIN- 
Mr^^V^'^^^O'' V'EWOR OPINIONS 
STATED DO NOT NECESSARILY REPRE- 
SENT OFFICIAL NATIONAL INSTITUTE OF 
EDUCATION POSITION OR POLICY 



A COMPARATIVE STUDY OF THE EFFECTIVENESS 
OF TWO BAYESIAN MODELS FOR PREDICTING THE 

ACADEMIC SUCCESSES OF SELECTED 
ALLIED HEALTH STUDENTS ENROLLED IN THE 

COMPREHENSIVE COrflUNITY COLLEGE 



BY 



DENNIS HINKLE 
VIRGINIA POLYTECHNIC INSTITUTE & STATE UNIVERSITY 
BLACKSBURG, VIRGINIA 



AND 



CHARLES A. HOUSTON 
VIRGINIA WESTERN COMMUNITY COLLEGE 
ROANOKE, VIRGINIA 



A PAPER PRESENTED AT THE 
AMERICAN EDUCATIONAL RESEARCH ASSOCIATION 
ANNUAL MEETING 
APRIL 1977 
NEW YORK 



•PfR MISSION TO PPPRODUCE THiS COPV- 
RiGHTEO MATERIAL HAS QEEN GHANTfcO BY 



Houston, Jr. 



TO ERIC ANO 0HGAMt2ATI0NS OPERATING 
UNOER AGREEMENTS WITH THE NATIONAL IN- 
STtTlJTE OF EOUCATION FURTHER REPRO- 
OUCTION OUTSIDE THE ERIC SYSTEM RE- 
QUIRES PERMISSION Of THE COPYRIGHT 
bV5»'NER ■ 



A CDMPARATIVE STUDY OF THE EFFECTIVENESS OF TWO BAYESIAN 
MODELS FOR PREDICTING THE ACADEMIC SUCCESSES OF 
SELECTED ALLIED HEALTH STUDENTS ENROLLED 
IN THE COMPREHENSIVE 
COMMUNITY COLLEGE 

by 

Charles Aiken Houston, Jr. 
(ABSTRACT) 

Because of high enrollment demands, limited instructional spaces, 
and the high cost of programs, many community colleges have been forced 
to place stringent admission requirements in the guidance and selection 
of certain allied health curricula such as Dental Assistant, Mental 
Health, Nursing, and Radiology. In this guidance-selection environment 
in which there were more candidates than vacancies, methods and/or 
procedures must be established that discern the "best" applicants. 
Since neither the classical statistical models which utilize correlations, 
regression, discriminate analysis, etc. nor the counselor-selection models 
have typically utilized all the information regarding a student, the need 
for more efficient and effective guidance-selection models was indicated. 
In this context Bayesian-type models have been proposed that can utilize 
the strengths of both the classical statistical models and the counselor- 
selection models. 

The purpose of the study was to present and evaluate Bayesian- 
type models for estimating probabilities of program completion and 
predicting first quarter grade point average (GPA) . Bayesian Model 1 — 
Estimating Probabilities of Program Completion was developed from the 
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discrete case of Bayes* formula with counselors* inputs as a priori 
probabilities and posterior probabilities of graduate status of the 
discriminant analysis function as likelihoods. The a priori probabilities 
and likelihood probabilities were combined in Bayes' Theorem to produce 
posterior probabilities of successful program completion. Bayesian Model 
2 --Predicting First-Quarter GPA*s which was an application and special- 
ization of the Bayesia'i linear model developed by Lindley and Smith (1972) 
involved the assumption of homogeneity of regression coefficients (but 
not intercepts) across groups. The efficiencies of the Bayesian-type 
models were compared and evaluated in teras of two counselor „se^ 
models and two classical statistical models. 

Although inspection by curriculum and by individual students 
indicated that all three program completion iitodels were appropriate in 
specific cases, an analysis of ranking efficiencies wsing the Friedman 
(1937) test rejected the hypothesis of identical treatments. Further 
analyses using multiple comparisons aspects proposed by Anderson (1959) 
indicated that there were statistically significant linear comparison 
differences among the models with the Bayesian Model 1 ranking as the 
most efficient model. An analysis of the three GPA models using an 
analysis of variance test contradicted the hypothesis that the three 
treatment effects were equal at the .01 -level with Tukey's Test for 
differences of average absolute-errors confirming that the Bayesian 
Model 2 and the Classical Statistical Model 2 were more efficient than 
the Counselor Model 2. The results of ihe study supported several recom- 
ntendations for further study. 



INTRODUCTION* 



Because of high enrollment demands, limited instructional spaces, 
and the high cost of programs, many cormiunity colleges have been forced 
tO' place stringent admission requirements in the guidance and selection 
of students for certajn allied health curricula such as Dental Assistant, 
Mental Health Technology, Nursing, and Radiology. Since recent research 
studies (A.W. Astin, 3975; H.S. Astin, 1970; Summerskill, 1962; Trent 
and Medsker, 1967) have reported withdrawal rates of 50 percent and 
higher for students entering conrounity col lege prbgra^^ a need for more 
efficient, and effective guidance-selection models was indicated. Models 
that accurately predicted first-quarter GPA's and efficiently estimates 
the probability of successful program completion would benefit both the 
college and the students that enroll in these curricula. 

The purpose of the study was to present and evaluate Bayesian-type 
models for estimating probabilities of program completion and predicting 
first-quarter GPA's for students entering four allied health curricula 
of the comprehensive conmunity college. Bayesian Model 1 ~ Estimating 
Probabilities of Progragi-Completion was developed from the discrete case 
of Bayes' formula as presented by Powers (1973, p. 4). Powers' 
methodology which utilized counselor/faculty advisors' inputs as priori 
probabilities was extended by using posterior probabilities of graduate 
status of the discriminant analysis function a,s likelihoods. 



*This study suimiarizes a dissertation submitted by the second author 
in partial fulfillment of the requirements for the degree of the Doctor of 
Philosophy in Educational Research and Evaluation, Virginia Polytechnic 
Institute and State University, April 1976, Blacksburg, Virginia. 
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The a priori probabilities and likelihood probabilities were combined 
in Bayes' Theorem to produce posterior probabilities of successful pro-, 
gram completion. Bayesian Model 2 — Predicting First-Quarter GPA's 
utilized a computer program developed by Kazuo Shigemasu (1975). This 
ROdel which was an application and specialization of the Bayesian linear 
model developed by Lindley and Smith (1972) involved the assumption of 
homogeneity of regression coefficients (but not intercepts) across 
groups. This methodology (first proposed by Novick, Jackson,. Thayer, 
and Cole, 1971) comparing GPA predictions across different schools 
with similar programs was adjusted in terms of comparing similar programs 
within one college (i.e. the m-groups were now different programs instead 
of different schools). The efficiencies of the Bayesian-type models 
were compared and evaluated In terms of two counselor selection models 
and two classical statistical models. 

Although a review of the literature dealing with the prediction 
of academic success revealed a wealth of materials concerning the prog- 
nosis of college and university success, there have been few studies 
that dealt with the problem in an. experimental fashion. In general, 
prediction studies have beten ex post facto research in which the inde- 
pendent variable or variables have already occurred and the researcher 
starts with the observation of a dependent variable or variables. Little 
or no efforts have been directed toward cross validation studies which 
predict the academic success of the next class and/or evaluate the 
efficiency of the prediction models in terms of the new class. The 
literature review also noted that little effort has been directed toward 
developing and evaluating multiple prediction procedures. Therefore, 
the purpose of the present study was to present and evaluate multiple 
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predictiion procedures in a quasi-experimental fashion by using data for 
new allied health classes first enrolled for Fall 197S. 

The design of the study required that three program-completion 
models and three GPA prediction models be constructed from data discerned 
from previous students enrolled from the Fall of 1972 through the 
Fall of 1974. These models were used to predict first-quarter GPA, 
and probabilities of program completion for students first enrolleji 
for Fall 1975. The efficiencies of the models were compared by examining 
predicted GPA in terms of actual first-quarter GPA and estimates of 
probabilities of program completion in tern.s of actual graduate-nongraduate 
status (enrolled or withdrawn) for Spring 1976. 

The total sample of the study consisted of 374 students who were 
enrolled in the curricula of Dental Assisting, Mental Health Technology, 
Nursing, and Radiology. Data for students enrolled from Fall 1972, 1973, 
and 1974 were used to develop statistical equations (discussed below) 
necessary for developing the .ix prediction models for students entering 
in the Fall 1975. 

The predictor (independent) variables consisted of selected academic 
variables discerned by examining both previous research results 
and present theories concerning academic achievement with additional 
recommendations from allied health counselors, faculty, and administrators. 
The criterion (dependent) variables consisted of (1) first-quarter GPA 
and (2) graduate/nongraduate status. 

Several statistical methods were employed in developing the 
six prediction models. These methods were summarized as follows: 
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Program-Completion Models 



Statistical Methods 



Bayesian Model 1 — Estimating 
Probabilities of Program- 
Completion 



Counselor Model 1 Estimating 
Probabilities of Program - 
Completion 

Classical Statistical Model 1 — 
Estimating Probabilities of 
Program-Completion 



GPA Prediction Models 

Bayesian Model 2 — Predicting 
First Quarter GPA*s 



Counselor Model 2 Predicting 
First Quarter GPA's 

Classical Statistical Method 
2 ~ Predicting First- 
Quarter GPA's 



Application of Bayes* formula 
using counselor's assignment of 
probability of successful programs 
completion as a priori probability 
and discriminate score of proba- 
bility of graduate classification 
as likelihood. 

Counselor's assignment of proba- 
bility of successful program 
completion. 

Discriminant analysis for graduate/ 
nongraduate group membership using 
the discriminate score to estimate 
the students' success in completing 
the specific allied health curricu- 
1 um. 



Bayesian M-group Regression 
Analysis with Identical Regression 
Coefficients 

Counselor's assignment of first- 
quarter GPA 

Forward stepwise multiple 
regression using SPSS REGR 



Several statistical methods were employed in comparing and evaluating the 
effectiveness of prediction models. The program completion models were 
evaluated by comparing the mean absolute-errors and the mean squared- 
errors by both individual curriculum and by combined curricula. The 
Friedman test was used to evaluate the three models by ranking the pre- 
dicted probabilities in terms of actual status. In the case that the 
null hypothesis of equal likely rankings within a block (student) was 
rejected, multiple comparisons were examined in terms of tests presented 
by Anderson (1959) for rank data. 
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The three GPA prediction models were evaluated by examining the 
absolute-error-loss funct^jon, squared -error-loss function, and using . 
the analysis of variance (F-test) of predicted and actual GPA differences 
for a single-factor experiment having repeated m&aaures on. the sa«e 
elcBont (Winter, 1962, pp. 105-124). The efficiencies of the models 
were compared and evaluated in terms of the average absolute-error-loss 
and average squared-error-loss by both individual curricula and combined 
curricula. 

The study was limited in that certain restrictions were placed on 
• the six prediction 'iiodels. This fact was especially true for the Classical 
Statistical Model 1 which employed multiple regression in the traditional 
sense because counselors' evaluations of students first enrolled Fall 
1972, 1973, and 1974 were unavailable to be used as independent variables 
in developing the GPA prediction equation. The study was also limited by 
certain time factors which required that the criterion of graduate/non- 
graduate status for students first enrolled Fall 1975 to be determined 
in terms of enrolled/not enrolled for Spring 1976. Other limitations 
concerned missing data of high school rank and School and College Ability 
Test (SCATS) scores. 

CONCLUSIONS 

The results of the study indicated that there were little dif- 
ferences in the prediction efficiencies of the three models in reducing 
the weighted average absolute-errors of the four allied health curricula. 
The analysis of rank order efficiencies of the three models indicated 
that the weighted average absolute-error differences between the actual 
.status (1 - graduate and 0 - non-graduate) and the predicted status for 
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the 93 students in the four allied health curricula were as follows: 

(1) Bayesian Model 1 (0.34) , (2) Counselor Model 1 (0.35) , and (3) 
Classical Statistical Model 1 (0.40). Further descriptive analysis of 
the rank order efficiences of the three models indicated that the 
weighted average squared-error differences between the actual status 

(1 - graduate and 0 - nongraduate) and the predicted status for the 93 
students in the four allied health curricula were as follows: (1) 
Classical Statistical Model 1 and Counselor Model 1 tied (0.22) and 

(2) Bayesian Model 1 (0.24). Although inspection by curriculum and by. 
individual students indicated that all three models were .appropriate 

in specific cases, an analysis of ranking efficiencies of the three 
models using the Friedman (1937) test rejected the hypothesis of identi 
cal treatment effects at the .05-level. Further analyses using multipl 
comparisons aspects proposed by Anderson (1959) revealed the following: 

1. There were no statistically significant differences 
between the rankings assigned to Counselor Model 1. 

2. There were statistically significant linear com- 
parison differences among models for rank 1. Inspection 
of the models receiving rank 1 clearly indicated that the 
Bayesian Model 1 (56 rank 1) exceeded the rank 1 of either 
the Classical Statiitical Model 1 (25 rank 1) or the Coun- 
selor ModeM (12 rank 1). 

3. There were statistically significant differences between 
linear rankings for Bayesian Model and Classical 
Statistical Model 1. Inspection of the data clearly 
supported the hypothesis that the Bayesian Model 1 

was the most efficient model in selecting successful 
allied health students who would complete their programs 
of study. 

A descriptive analysis of the three GPA models indicated there 
were little differences in the prediction efficiencies of the Bayesian 
and classical .statistical models in reducing the weighted average 
absolute-errors for the 93 students enrolled in the four allied health 
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curricula. Counselor Model 2 was li&ss efficient than either the Bayesian 
or classical statistical models. The rank order efficiencies of the three 
models indicated that the weighted average absolute-error differences 
between actual first-quarter GRA and predicted GPA were as follows: 
(1) Bayesian Model 2 (0.719), (2) Classical Statistical Model 2 (0.723), 
and (3) Counselor Model 2 (0.S29). An analysis of variance of mean 
differen;:es of absolute-errors using a single-factor design having repeated 
measures on the same elements F(2,184) = 5.774, contradicted the 
hypothesis that the three treatment effects were equal at the .01 level with 
Tukey's Test for differences of average absolute-errors confirming that 
the mean absolute-errors of the Counselor Model 2 differs from the 
means of both the Bayesian Model 2 and the Classical Statistical Model 2. 
Further descriptive analysis of weighted mean averages of predicted and 
actual GPA differences of squared-error found little differences in the 
predicted efficiencies of thfe three models. A rank order of most 
efficient to least efficient for the weighted average squared-errors was 
as follows: (1 ) Sayesian Model 2 (1 .206) , (2) Classical Statistical Model 
"2 (1 .233), and (3) Counselor Model 2 (1.368). An analysis of variance of 
mean differences of oquared-errors using a single-factor design having 
repeated measures on the same elements failed to reject the hypothesis 
that the squared-error differences were different at the .05 level for 
the three GPA prediction itiDdels, . 

Because several investigators (Nicholson, 1.970; Heist, 1968; 
Savicki and others, 1970) have noted that dropouts were not different 
from successful persisters on predicted GPA, the study concluded that 
there was a greater need to learn more about the prediction of graduation 
from college as a criterion of college success rather than college GPA. 

' n 



This fact Was especially true for the allied health curricula because 
in mst cases graduation itself was the key to entering the occupational 
fields. In order to provide the greatest information to counselors and 
students concerning an estimate of their predicted graduate/nongraduate 
status, the need for a multiple comparison selection system was established 
by the results of the present study. These results indicated that the 
counselor and possibly the student could benefit from examining the 
prediction estimates provided by the different program-completion models. 
Because graduation was partially a function of GPA, the study concluded 
that GPA*s should be viewed as necessary but not sufficient conditions 
for future academic successes. Therefore, it was necessary to examine 
predicted GPA '5 in terms of predicted program-completion probabilities. 

RECOMMENDATIONS 

The results of the study supported several recommendations for 
further study. First and foremost, the Bayesian-type procedures produced 
results which indicated that they are deserving of further investigating. 
Since the major problem in Bayesian inference has been that of quanti- 
fying a priori information, ^further studies were recommended in terms of 
establishing methodologies that assist counselors and faculty advisors 
in quantifying and understanding the processes of estimating and assigning 
probabilities for program-completion and predicting first-quarter GPA's. 
Additional investigations should discern topics such as: How did these 
counselors actually make the decisions for assigning the program-completion 
es t imates and the GPA7 pred i c ti ons ? Why were the mode Is very accurate in 
the case of student 5 and totally inaccurate in the case of student 10? 
Were there characteristics that can be identified which would discriminate - 
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between students who were accurately classified and students who were 
inaccurately classified? 

In terms of the GPA modeis, it was certainly obyious that addi- 
tional variables should be added to the Classical Statistical Model 2 
and the Bayesian Model 2. It was recommended that additional academic 
variables with biographical, demographical , and attitude/opinion 
variables be added to the study. The possibility of using certain types 
of factor analyses of the variables and using the factor scores as 
independent variables should be discerned. The use of additional academi 
and nonacademic variables for the program-completion models r;hould also 
be investigated. In addition, studies that jointly examine GPA and 
problem-completion prediction models and their relationship should be 
examined. 

Although the three program-completion and GPA models represented 
extremes on a continum, it was recommended that the following additional 
models be proposed anii investigated in terms of multiple comparison 
studies: - 

1. Both the Classical Statistical Model 2 — Predicting First- 
Quarter GPA and the Bayesian Model 2 — Predicting First-Quarter 
GPA should be extended with the inputs of certain counselors' 
evaluations as independent variables. 

2. The use of dummy independent vaHables in coding the four 
allied health programs for the Classical Statistical Model 2 ~ 
Predicting First-Quarter GPA should be considered in future 
studies. 

3. The use of certain nonlinear transformations should be 
considered in developing future GPA prediction models. 

4. Novick's original M-group regression program should be 
considered in future studies. 

5. Theoretical work should be extended to incorporate a priori 
beliefs of counselors into the Bayesian H-group regression 
procedures. 
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In addition many cross validation studies should be examined in order to 
establish the validity of impressive correlation coefficients that have 
been developed by ex post facto research. 
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